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ABSTRACT 

In  some  acquisition  procedures  of  military  equipment  there  are  no  crisp  data  available  and/or  no 
willingness  to  provide  quantitative  data.  This  might  often  occur  in  an  early  stage  of  the  procedure,  but 
might  also  be  a  characteristic  of  the  qualitative  nature  of  some  or  all  the  criteria  involved  in  the 
assessment  of  the  alternative  equipments  to  be  selected  or  ranked.  In  such  a  case  we  propose  the  use  of 
an  ordinal  outranking  method  of  the  “ORESTE”  type,  which  requires  only  ordinal  data  and  rankings  of 
criteria  according  to  their  importance. 

We  discuss  how  to  use  such  a  method  in  the  context  of  an  acquisition  procedure  of  military  equipment, 
and  how  to  conduct  the  incomparability  analysis.  A  visualization  of  the  incomparabilities  and  of  the  total 
and  partial  rankings  is  presented. 

Finally  we  discuss  the  combined  use  of  an  ordinal  method  with  other  outranking  methods  and  the 
interpretation  of  the  results  in  the  context  of  the  acquisition  procedure  of  military  equipment. 

INTRODUCTION 


In  the  S AS -080  (NATO  S AS -080  Specialist  Meeting  -  Brussels  22-23  October  2009)  contributions  on 
“Using  an  outranking  method  supporting  the  acquisition  of  military  equipment”  (SAS-080  14)  and  on 
“Assessing  and  Visualizing  Incomparabilities  by  using  an  outranking  method  supporting  the  acquisition  of 
military  equipment”  (SAS-080  15),  we  reminded  that  outranking  methods  for  multicriteria  decision  aid 
belong  typically  to  the  so-called  European  School  of  Multicriteria  Decision  Making  (MCDM),  which 
came  into  existence  with  the  stimulating  work  of  B.  Roy  ([13], [14], [15], [16]).  We  will  repeat  here  some 
of  the  points  addressed  in  these  papers.  The  outranking  approach  is  based  on  a  fundamental  partial 
comparability  axiom  where  incomparability  is  a  key  concept  ([5],  p.80).  In  contrast  with  this  approach 
there  is  the  so-called  American  School  in  which  Th.  Saaty  plays  an  important  role  with  his  “Analytical 
Hierarchy  Process”  (AHP  Method)  in  which  there  is  no  place  for  incomparabilities  [17].  In  the  European 
School  we  think  that  incomparabilities  between  alternatives  to  be  ranked  or  to  be  selected,  are  a  natural 
aspect  of  any  MCDM  problem,  in  which  criteria  evaluating  the  performance  of  these  alternatives  are 
conflicting  -  meaning  that  for  instance  two  different  criteria  can  have  inversed  preferences  between 
couples  of  the  same  alternatives.  If  this  happens  on  a  large  set  of  couples  of  criteria,  then  we  claim  that 
neglecting  these  conflicts,  is  leading  to  decisions  which  are  often  far  from  the  original  data  of  the  MCDM 
problem.  Although  the  final  objective  in  practice  is  to  decide  about  a  ranking  or  about  a  selection  of  a 
subset  of  the  alternatives,  we  claim  that  the  decision  maker  should  be  supported  by  methods  which  are 
warning  about  the  presence  of  incomparabilities.  We  even  claim  that  it  should  be  possible  to  assess  the 
importance  (the  intensity)  of  these  incomparibilities  in  order  to  fully  inform  the  decision  maker  about  it, 
before  the  final  decision  is  made. 
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Using  an  Ordinal  Outranking  Method 
Supporting  the  Acquisition  of  Military  Equipment 


Many  different  methods  belong  to  the  outranking  class.  For  overviews  we  refer  to  [5],  [18]  and  [19].  For  a 
detailed  description  of  industrial  applications  with  the  oldest  member  (ELECTRE)  of  this  class  we  refer  to 
[11].  In  this  paper  we  will  concentrate  on  the  well-known  ORESTE  method  ([10], [12])  which  is 
complementary  to  the  PROMETHEE  methods.  There  are  other  methods  belonging  to  the  European 
School  like  MACBETH  [1]  which  in  SAS-080  is  the  subject  of  a  keynote  address  by  C.  Bana  e  Costa. 

The  ORESTE  method  is  sufficiently  well-known  by  System  Analysis  specialists,  to  skip  in  this  paper  all 
mathematical  aspects.  For  details  we  refer  to  [10]  and  [12]  . 

The  ORESTE  method  has  sometimes  been  used  in  the  eighties  and  nineties  of  last  century  by  small  teams 
of  Belgian  MoD  equipment  acquisition  services  as  a  complement  to  the  PROMETHEE  methods.  This 
MCDM  method  is  taught  in  the  curriculum  of  the  High  Staff  College  for  Military  Administrators  of  the 
Belgian  MoD.  Currently  personnel  involved  with  equipment  acquisition  can  use  these  methods  on  an 
individual  basis. 

In  this  paper  we  concentrate  on  practical  features  of  the  ORESTE  method  especially  related  to  the 
incomparability  analysis ,  the  typical  use  for  military  equipment  acquisition,  and  we  illustrate  the 
discussion  primarily  by  an  implementation  we  called  MCDMTool  [7]. 

INPUT  DATA 

We  start  with  the  same  data  as  in  the  paper  SAS-080  on  “Using  an  outranking  method  supporting  the 
acquisition  of  military  equipment”,  but  now  we  intend  to  apply  the  ORESTE  method  for  the  sub-problem 
related  to  the  criterion  assessing  “Human  Resources”  issues. 

This  is  illustrated  in  Figure  1  with  MCDMTool  in  which  we  will  now  only  look  at  the  sub-tree  “Human 
Resources”. 
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Figure  1 :  Input  structure 

All  assessments  of  alternatives  for  all  criteria  are  ordinal  (represented  by  ranks).  The  relative 
importance  of  the  criteria  are  also  given  by  ranks. 

This  is  illustrated  in  Figure  2  with  MCDMTool. 
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Figure  2:  Input  ordinal  data 
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Figure  3:  Ranks  of  Alternatives  for  criteria 


ORESTE  I  COMPUTATIONS 


First  all  ranks  are  converted  into  Besson  ranks  [10]  to  take  into  account  possible  ties  (in  this 
example  there  are  none).  The  results  are  in  Figure  4. 
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Figure  4:  Besson  ranks 


Then  a  first  aggregation  of  the  two-dimensional  alternatives-criteria  data  (Figure  2)  is  performed  by 
computing  for  each  couple  alternative-criteria  as  “distance”  score  to  and  ideal  position  occupied  by  the 
best  alternative  for  the  most  important  criterion.  This  score  is  the  mean  value  of  the  Besson-rank  rc  of 


criterion  c . ,  and  of  the  Besson-rank  r  ( a )  of  the  alternative  a  in  criterion  c . .  This  mean  value  can  be 

j  cj  j 


computed  by  d(a,Cj)  = 


with  Rm  D 


The  parameter  R  can  be  adapted  for  two 


different  parts  of  the  input  data  table.  Most  users  will  take  the  default  value  R=  1  (see  Figure  5).  Also  the 
coefficient  Vi  can  take  values  A  and  (1-A)  for  two  different  parts  of  the  input  data  table  .  Most  users  will 
however  take  the  default  value  A=l/2  . 


RTO-MP-SAS-080 


16-5 


Using  an  Ordinal  Outranking  Method 
Supporting  the  Acquisition  of  Military  Equipment 


Figure  5:  Parameter  setting 
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Figure  6:  “distance”  scores 


Then  all  these  “distance”  scores  are  converted  into  Besson-ranks  in  order  to  keep  the  method  fully  ordinal. 
We  call  these  Besson-ranks  global  ranks  pc  (a)  which  are  upper-bounded  by  the  value  mk  with  m  the 

number  of  alternatives  (here  10)  and  k  the  number  of  criteria  (here  4). 

These  global  ranks  are  illustrated  in  Figure  7. 
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Figure  7:  Global  ranks 


Then  we  compute  the  “mean”  rank  for  each  alternative  by  r(a)  =  (a)  which  is  actually  only  the 

j 

numerator  of  the  mean  of  all  the  global  ranks  the  alternative  a  got  in  the  previous  table.  The  values  of 
these  r(a )  are  in  Figure  8. 


Figure  8:  Mean  ranks 


Finally  the  alternatives  are  ranked  in  increasing  order  of  the  mean  ranks.  This  yield  for  our 
example  the  representation  in  Figure  9  together  with  the  profiles  of  the  alternatives  in  Figure 


10. 
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Figure  9:  Final  ranking 
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Figure  10:  Profiles 

ORESTE II  COMPUTATIONS 

Now  we  compute  for  each  couple  of  alternatives  a  and  b  two  preference  intensities 
C'(a,b )  and  C\b,a) 

C\a,b)  =  X  Pj(.b)-Pj(a) 

j:Pj(b)>Pj(a ) 

with  _ 

C  \b,  a)  =  X  PjUD-pjib) 

j'-Pj  (a)>Pj  (b) 


These  preference  intensities  are  upper-bounded  by  k 2  (m  —  1)  .  (see  [1 0]) 

When  we  divide  the  preference  intensities  by  this  upper  bound,  we  obtain  normalized 
preference  intensities  C(a,b)  and  C(b,a)  similar  to  the  preference  indicators 
n(a,b)  and  ^(b,a)  of  the  PROMETHEE  methods  (see  [4]  and  SAS-080  14  and  SAS-080 
15).  These  preference  intensities  are  in  Figure  11. 
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Figure  11:  Preference  intensities 


These  normalized  preference  intensities  are  then  used  as  coordinates  of  points  representing  couples  of 
alternatives  in  an  IPR-diagram.  For  a  parameter  set  defining  the  I,  P  and  R  regions  (see  [10])  this  yields 
then  a  set  of  couples  for  which  a  warning  is  triggered,  announcing  an  incomparability  if  the  point  is 
located  in  the  (red)  R-zone  of  the  IPR-diagram. 

The  result  for  a  very  risk-averse  parameter  setting  (meaning  that  incomparability  is  announced  even  for 
low  intensities  of  the  incomparability)  is  given  in  Figure  12. 

The  intensity  of  the  incomparability  can  be  measured  by  the  distance  of  the  point  representing  the  couple 
in  the  IPR-diagram,  to  the  bisector  of  quadrant  of  the  IPR-diagram.  This  distance  can  on  its  turn  be 
measured  by  the  parameter  Gamma  (/)  for  which  MCDMTool  provides  an  interactive  analysis 
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Figure  12:  IPR  analysis 


RTO-MP-SAS-080 


16-11 


Using  an  Ordinal  Outranking  Method 
Supporting  the  Acquisition  of  Military  Equipment 


Figure  13:  IPR  diagram 


Notice  in  Figure  13  that  for  instance  the  point  41  is  in  the  incomparability  region,  and  indeed  the  couple 
Eqt7  and  Eqt9  is  announced  by  the  IPR  analysis  as  incomparable. 

In  Figure  14  we  illustrate  the  use  of  reference  (standard)  profiles  to  determine  the  parameter  gamma.  This 
gamma  determines  the  incomparability  zone  and  indirectly  gives  a  measure  of  the  distance  of  the  point 
representing  a  couple  of  alternatives  to  the  bisector  (axis  of  symmetry  of  the  R-region)  of  the  quadrant  of 
the  IPR-diagram. 

This  is  interactive  analysis  for  two  different  values  of  gamma  is  illustrated  in  Figure  15. 

It  shows  for  instance  that  the  couple  Eqt6  and  Eqt8  (represented  by  the  point  37  in  the  IPR-diagram)  has 
roughly  the  same  incomparability  (conflict)  intensity  than  the  couple  Eqt2  and  Eqt3  (represented  by  the 
point  10  in  the  IPR-diagram)  because  these  points  are  almost  on  the  same  “gamma”-distance  from  the 
bisector  of  the  quadrant.  However  it  is  much  more  critical  and  more  important  to  trigger  the  warning  of 
incomparability  for  Eqt6  and  Eqt8  than  for  the  other  couple,  because  the  couple  is  among  the  top-ranked 
in  ORESTE  I.  This  means  for  instance  that  it  is  very  relevant  to  have  this  analysis  tool  in  the  case  when 
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this  MCDM  problem  consists  of  the  selection  of  two  alternatives  among  ten.  In  that  case  the  choice  of  the 
second  alternative  to  be  selected  (after  the  overall  winner  Eqtl)  is  a  hard  decision  to  be  taken,  which  might 
require  some  further  investigation  (re-assessments,  additional  criteria,. . .). 


H  Parameters  forHuman  Resources  n*  S 

Sub-criteria  integration 

ORESTE  ▼ 

Beta  Jo.OS 

Determine 

c*  |o. 125 

Determine 

Gamma  |o  5 

Determine 

A  inf  |o.5  |  R  inf  1 1 . 0 

A  sup  |o.5  |Rsu|>  1.0 

Apply 

Close 

Figure  14:  Interactive  determination  of  gamma  for  reference  (standard)  profiles 
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Gamma 
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Figure  15:  Interactive  assessment  of  the  incomparability  intensity 
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CONCLUSION 

This  ordinal  outranking  method  ORESTE,  implemented  with  MCDMTool,  is  a  powerful  and  relevant 
analysis  tool,  providing  the  decision  maker  with  the  necessary  information  in  the  case  of  hard  decisions 
about  expensive  military  equipment  to  be  taken  (or  advise  to  be  given)  in  the  context  of  conflicting  criteria 
leading  to  incomparable  alternatives  (especially  on  the  top  of  the  ranking).  This  is  the  main  reason  why 
we  integrated  it  into  the  MCDMTool  software. 
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